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This article presents a review of the literature over the past 10 years into the 
use of technological interventions that tutors might use to encourage students to 
engage with and action the feedback that they receive on their assessment tasks. 

The authors hypothesise that technology has the potential to enhance student 
engagement with feedback. During the literature review, a particular emphasis 
was placed on investigating how students might better use feedback when it is 
published online. This includes where an adaptive release technique is applied 
requiring students to submit an action plan based on their feedback to activate 
the release of their grade, and electronic generation of feedback using statement 
banks. Key journals were identified and a snowball technique was used to select 
relevant literature. The use of technology to support and enhance student learn¬ 
ing and assessment is well documented in the literature, and effective feedback 
practices are similarly well published. However, in terms of the use of technol¬ 
ogy to support and enhance feedback processes and practices (i.e. production, 
publication, delivery and students making use of feedback through technology), 
we found the literature to be limited. 
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Introduction 

Feedback on assessments is an integral feature of effective and efficient teaching 
and learning, and can be one of the most powerful ways in which to enhance and 
strengthen student learning (Black and Wiliam 1998; Sadler 2010). Price et al. 
(2010, 277) argue that feedback is “the most important part of the assessment pro¬ 
cess”. Feedback enables learning by providing information that can be used to 
improve and enhance future perfonnance. This paper explores current literature 
around the appropriate use of technology in the production and delivery of feed¬ 
back. It was anticipated that the literature would support the hypothesis that tech¬ 
nology has the potential to enhance student engagement with feedback. 1 

Models, principles and conditions that support good feedback practice have been 
proposed (for example, Gibbs and Simpson 2004; McDowell et al. 2005; Nicol and 
Macfarlane-Dick 2006). For feedback to be effective and attended to by students, it 
needs to be timely (i.e. while it matters to them and useful for future assessments), 
legible, and aligned with explicit assessment criteria, as emphasised by Nicol (2009, 
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337): “Feedback should be of sufficient quantity; timely; it should focus on learning 
not marks; it should be related to assessment criteria and be understandable, attended 
to and actually used by students to make improvements on their work”. Appropriate 
use of learning technologies can encourage better engagement with feedback. 

Despite the promotion of good feedback, much of the practice reported tends to 
focus on the ineffectiveness of feedback (Bloxham and Boyd 2007; Flounsell 2008; 
Rowe and Wood 2007; Rust, O’Donovan, and Price 2005). Pressures in the UK 
higher education sector (Department for Education and Skills 2003) resulting in 
modularisation and semesterisation have seen the bunching of assessment tasks 
toward the end of the module. This not only limits the scope for feedback practices 
that feed forward into future assessments (Fliggins, Flartley, and Skelton 2002; Irons 
2008; Price and O’Donovan 2008; Yorke 2001), but also into their professional 
practice (Poulos and Mahony 2007). These pressures have also seen the writing of 
feedback under increasingly tight time constraints (Chanock 2000). The result has 
been a negative impact on the student experience of feedback as supported by 
responses to the National Student Survey (Higher Education Funding Council for 
England 2007). Students have expressed dissatisfaction with the adequacy of the 
feedback they receive both in terms of timing and usefulness (Mutch 2003; Price 
et al. 2010). This is echoed further by large-scale (Hounsell and Entwistle 2007) 
and small-scale (Crook, Gross, and Dymott 2006) studies into the student experi¬ 
ence of assessment and feedback. 

Staff complain that feedback does not work (Weaver 2006) and that students do 
not act on feedback (Mutch 2003). It is claimed that students are only concerned 
with their grades (Wojtas 1998; Nesbit and Burton 2006), see feedback as a means 
to justify the grade (Price and O’Donovan 2008; Price et al. 2010) or only read the 
qualitative comments if the quantitative mark is outside of their expectations 
(Duncan 2007). Some authors have argued that student disengagement with feed¬ 
back is based on sceptical or anecdotal evidence from tutors (Carless 2006; Higgins, 
Hartley, and Skelton 2002; Weaver 2006), and Burke (2009, 41) wonders whether 
“students interpret the term ‘feedback’ literally and use it only to look back on 
work they have completed, and are not aware or able to use tutor comments to 
‘feed-forward’ and contribute to their ongoing development”. In addition, Maclellan 
(2001, 316) suggests that use of ‘implicit criteria’ by tutors means that students do 
not view feedback on their learning as helpful, and Bloxham and Boyd (2007) and 
Higgins, Hartley, and Skelton (2002) point to difficulties for students in deciphering 
handwritten feedback comments as a further reason why students do not read or 
make use of their tutors’ comments. 

There is a growing body of research exploring how technology might be used to 
support effective and efficient feedback practices. Nicol (2009) explains the aims of 
the Re-Engineering Assessment Practices in Scottish Education project and 
demonstrates the ways in which technology can be used to support the development 
of self-regulation, the organisation of assessment tasks and the provision of feedback 
in large (150-900 students) first-year classes, where modularisation and larger stu¬ 
dent numbers have reduced fonnative assessment and feedback opportunities. 


Approach 

This article builds on an earlier literature review undertaken and made available via 
a wiki by the authors in 2008/09 in preparation for a research project to explore 
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how students at Sheffield Hallam University engage with feedback delivered elec¬ 
tronically when three specific technological interventions are used. The research 
project (Parkin et al. forthcoming) aimed to find out how students might engage 
with feedback and formulate actions to improve future learning when: 

feedback and grades are published online; 

feedback is published online without the grade and using an adaptive release process 
whereby students are encouraged to reflect on and submit an action plan based on 
their feedback to activate the release of the grade; and 

feedback is produced via electronic comment banks aligned to the original assessment 
criteria and presented in a table or grid format. 

In this article, recent studies are considered alongside the previous literature in pro¬ 
viding an up-to-date review of the technological interventions that might encourage 
student engagement with feedback. 

A literature search was undertaken using the terms ‘technology’, ‘feedback’ and 
‘engagement’ across electronic catalogues and search engines. In addition, the con¬ 
tents of the Association for Learning Technology’s Journal , Research in Learning 
Technology, Assessment & Evaluation in Higher Education and British Journal of 
Educational Technology (identified as being three of the key journals with a focus 
on the use of educational technology and assessment in higher education) were 
browsed from 2001, as well as relevant conference proceedings including the Asso¬ 
ciation for Learning Technology’s Conference and the International Computer- 
Assisted Assessment Conference. Articles were selected based on both the title of 
the paper and its abstract, and we used the ‘snowball’ technique to identify addi¬ 
tional sources from references within the articles. 

Our focus is on the production, delivery and the process for making use of that 
feedback rather than the format of the feedback itself. We were particularly inter¬ 
ested in research and discussion that explored the publication of feedback and 
grades online, the automated adaptive release of grades (or how technology can 
support encouraging student engagement through disengaging the grade from feed¬ 
back), and the use of technology to automatically generate feedback aligned to 
assessment criteria through pre-populated statement banks. 

This literature review aims to address the comment by Burke (2009, 42) in 
which the author claims “it appears that we currently have a blind spot in relation 
to strategies for students making effective use of feedback” and supports Whitelock 
(2009, 199): 

e-Assessment has the potential to offer new fonns of assessment with immediate 
feedback to students ... It is therefore becoming increasingly important to construct a 
pedagogically driven model for e-assessment that can incorporate e-assessment and 
e-feedback into a holistic dialogue learning framework, which recognises the 
importance of students reflecting upon and taking control of their own learning. 


Findings 

Our findings have been presented within the technological interventions identified 
in the approach. Two additional sections have also been included regarding the use 
of technology to encourage student engagement with feedback as there are a wide 
range of articles focused on the use of computer-assisted assessment with instant 
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feedback and a growing use of peer assessment supported by electronic or online 
tools. It felt worth capturing this wealth of literature. 


Using technology to publish feedback 

Sending tutors’ comments electronically (Bloxham and Boyd 2007; Crossouard and 
Pryor 2009; Denton 2001a, 2001b, 2003; Denton et al. 2008; Gipps 2005; Irons 
2008) is an effective and simple means of communicating formative feedback to 
students, and can enhance the way in which students receive and engage with feed¬ 
back. This may be feedback on an individual or group piece of work, or feedback 
generic to an entire student cohort on a module. Students can receive their feedback 
in privacy, enabling them to respond to their feedback in different ways and at dif¬ 
ferent times (Price and O’Donovan 2008). A number of other studies have reported 
on the greater impact of electronic or online feedback (Van den Boom et al. 2004; 
Guardado and Shi 2007; Tuzi 2004). 

Price and O’Donovan (2008) argued that feedback should be incorporated into 
the learning and teaching process both to improve student engagement with feed¬ 
back and to enable the effectiveness to be measured. Maclellan (2001) argued that 
students should be monitoring their own perfonnance in order to make effective use 
of feedback to generate improvement in learning, and this has been supported by 
Carless (2006, 229) who suggested that students should be provided with the 
“means to distinguish accurately their achievements in different assignments”. Fulda 
(2005) reports on US colleges and universities mandating that grades be entered 
online. Not only does this offer cost and efficiency savings in transcribing grades 
and reducing the risk of error, the same interface can be made available to students 
allowing them quicker access to their grades. 


Producing feedback electronically 

Examples of producing feedback electronically include the use of ‘Track Changes’ 
and ‘Comments’ in Microsoft Word to alter and annotate the student’s original 
word-processed work (Crossouard and Pryor 2009), and comments typed in a sepa¬ 
rate document or digital ink using a tablet PC (Plimmer and Mason 2006) providing 
individual feedback on student work. Heinrich et al. (2009) reported on the use of 
technology in the context of managing and marking assignments for a large student 
cohort. It was claimed that the real impact of technology is in administration effi¬ 
ciencies (e.g. documents are easily accessible to all involved in the marking process 
any time and any place). As such this frees up time for focusing on producing qual¬ 
ity, more detailed and legible feedback, with the option to provide direct links to 
further online resources, articles and books. 

Recent practice and studies have focused on the development of systematic 
methods to produce and return feedback that has been mapped against predefined 
assessment criteria or learning outcomes (Irons 2008; Sadler 2010). The use of 
Microsoft Office applications, templates and the computer-supported generation of 
individualised feedback documents from statement or question response banks have 
been developed to improve quality in response to increased student numbers 
(Denton 2001a, 2001b, 2003; Denton et al. 2008; Heinrich et al. 2009; Hepplestone 
and Mather 2007; Irons, 2008). Homby (2005) reports on the use of checklists and 
statement banks to automate the feedback process using a criterion reference 
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scheme in which grade related statements were generated and cut-and-pasted into a 
personalised feedback report for each student and returned electronically. 

A recent innovation is the use of audio feedback (Middleton 2009), although to 
date studies and practice have remained small-scale despite personal recording 
devices being readily available and recording software relatively easy to use. In a 
recent study by Rodway-Dyer, Dunne, and Newcombe (2009) of first-year students 
receiving audio feedback, they found that the main advantage was the greater detail 
and depth of information offered, as well as removing misunderstandings due to 
illegible handwriting. A major disadvantage of the approach is that it often sepa¬ 
rates the assignment and feedback with loss of annotations (Ribchester, France, and 
Wheeler 2007), making it difficult for students to identify the point in their work 
being discussed (Rodway-Dyer, Dunne, and Newcombe 2009). Rotheram (2009) 
advocates the use of audio feedback, claiming that students have the advantage of 
replaying feedback, although Rodway-Dyer, Dunne, and Newcombe (2009) found 
no evidence to suggest that students will no more listen repeatedly to audio feed¬ 
back than they would re-read written feedback. Merry and Orsmond (2008) found 
that all students listened to the feedback with a copy of their submitted work in 
front of them at least once, with the majority of students altering their work as they 
listened to the feedback. In a further study of first-year and third-year students by 
Ribchester, France, and Wheeler (2007) it was found that the length of the record¬ 
ing may impact on student engagement: the first-year students had a natural listen¬ 
ing threshold of around five minutes while the third-year students listened to the 
whole recording. Despite claims by Merry and Osmond (2008) and Rotheram 
(2009) that audio files are easy to store, and the findings by Ribchester, France, and 
Wheeler (2007) that students accessed the feedback off-campus, there was a reluc¬ 
tance by students to store feedback on personal devices (Ribchester, France, and 
Wheeler 2007). 


Adaptive release of grades 

We were interested in how technology might be used to encourage students to 
engage with their feedback by initially providing feedback without the grades, and 
only revealing the grade once the student had completed a task to indicate that they 
had read, considered and sought the opportunity to use the feedback in their future 
learning. Although we were unable to find examples from the literature to support 
this practice through the use of technology, several authors have advocated that dis¬ 
engaging the grade or withholding the grade from feedback promotes student learn¬ 
ing (for example, Boud and Falchilcov 2006; Butler 1988; Carless 2006; Ecclestone 
and Swan 1999; Nicol 2007; Potts 1992; Rust, O’Donovan, and Price 2005). Such 
practice also reflects the view that feedback can only support learning if it involves 
the production of evidence or is built into personal development planning (for 
example, Bloxham and Boyd 2007; Draper 2009; Higgins, Hartley, and Skelton 
2002; Irons 2008; Mutch 2003; Rust, O’Donovan, and Price 2005). 


Computer-assisted assessment 

Although not a focus of the original literature review, it was apparent that one of 
the most published uses of technology to provide students with formative feedback 
(or e-feedback) is through computer-based testing, quizzes or assessment using mul- 
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tiple-choice or similar objective question types (Bull and McKenna 2004; Denton 
et al. 2008; Handley and Cox 2007). The purpose is to promote independent learn¬ 
ing and enable students to monitor their progress and development by undertaking 
assessment tasks at a time and place of their choosing (Irons 2008). Challis (2005) 
promotes the opportunity computer-aided testing allows for students to effectively 
leam and explore areas of perceived weakness in privacy without the fear of reveal¬ 
ing mistakes to peers or tutors, while Miller (2009) discusses the use of computer- 
assisted assessment to provide feedback to improve student learning. Such tests are 
often made available as open access, essentially allowing students to repeat the 
assessment an unlimited number of times. However, despite their formative nature, 
the results of these tests may contribute towards a summative grade, and are there¬ 
fore initially made available as a single attempt to generate the grade, and subse¬ 
quently made openly available at the end of the module to aid revision. The use of 
commercial software products can deliver detailed formative feedback for each indi¬ 
vidual question, or specifically tailored to a particular response, more efficiently 
than is possible with traditional assessment (Brown, Bull, and Race 1999; Bull and 
McKenna 2004; Gipps 2005), and students favour the immediacy of such feedback 
as it keeps the activity and result closely connected, meaningful and current 
(Channan 1999; Denton et al. 2008). Peat and Franklin (2002) describe the devel¬ 
opment of various online computer-based assessment components (including the use 
of weekly quizzes, mock examinations and self-assessment modules) and how stu¬ 
dents respond to these changes, while Homby (2005) discusses the efficiencies 
afforded by the use of automated computer assessment in allowing students to take 
control of their learning and self-assess. Computer-adaptive testing (Challis 2005; 
Conole and Warburton 2005; Irons 2008) can be used in order to match the forma¬ 
tive assessment to the individual students’ ability as well as setting benchmarks for 
students to attain before moving on to the next learning or assessment activity. 


Peer assessment 

Technological innovations and developments have been used to engage students in 
the process of peer assessment and peer feedback (Freeman and McKenzie 2002; 
Li, Liu, and Steckelberg 2010; Loddington et al. 2009). Both Freeman and 
McKenzie (2002) and Loddington et al. (2009) illustrate that the use of technology 
can improve the fairness of group assessment by acknowledging, rating and reward¬ 
ing individual contributions and enhancing students’ learning from team tasks, 
whereas Li, Liu, and Steckelberg (2010) found a significant relationship between 
the quality of the peer feedback the students provided for others and the quality of 
the students’ own final projects. 


Conclusions 

This literature review found that there are a significant number of publications and 
articles that promote and suggest good feedback principles, models and practice. 
Similarly, a wide range of literature focuses on the belief that feedback practice in 
its present form does not work. In some cases the authors seek answers by criticis¬ 
ing current structures in higher education (i.e. modularisation, large student numbers 
and time pressures) for feedback that does not feed forward into future modules or 
assignments. Also some literature seeks to blame students for focusing only on the 
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grade, not seeing the connections between modules and assessments, or concentrat¬ 
ing on feedback comments only when the grade does not meet their expectations 
for effort put into completing the assessment. 

A growing number of studies support the hypothesis that technology has the 
potential to enhance student engagement with feedback, suggesting that changing 
the process by which feedback is made available to students can enhance student 
engagement with feedback. Feedback published online offers students a level of 
flexibility in being able to read it at a time convenient to them and concentrate 
more deeply on the comments in the absence of their peers. As the feedback is 
stored online, students are offered further convenience in accessing this information 
whenever and wherever they complete future assessments, and where available via 
a virtual learning environment, for example, the feedback can be used within the 
context of the rest of their learning. Where grades are aggregated alongside this 
feedback, students can use this quantitative infonnation to inform their performance 
in future assessment tasks. There is further evidence to suggest that feedback 
returned electronically is timelier as time is saved during administrative processes. 

Publishing feedback online can often require a change in the way feedback is 
produced. The use of typed comments on students’ original work enables students 
to see legibly the areas requiring focus and attention. The use of electronic state¬ 
ment banks to generate feedback aligned to the original assessment criteria, often 
used to speed up the generation of feedback, enables students to view their 
strengths and weaknesses in a structured and cohesive way and use these to set 
future learning targets. The growing use of audio feedback enables tutors to provide 
more detailed feedback than traditional written methods, although it is important to 
consider that students’ attention spans may diminish during listening to lengthy 
recordings. In using both statement banks and audio feedback, it is essential that 
any points raised are referred in some way back to the students’ original work, 
otherwise the feedback may appear separate and unhelpful. 

There is evidence that providing feedback without the grade encourages students 
to focus on the actual comments. We were interested in identifying methods used, 
in particular the use of technology, to encourage students to reflect on and develop 
action plans as a result of receiving their feedback without the grade, only receiving 
their grade after acting on their feedback. Although we could not find any evidence 
of the process being supported electronically, there were suggestions that student 
behaviour on making use of feedback could be changed through linking the process 
into personal development planning. 

While reviewing the literature, two further uses of technology to support student 
engagement with feedback were identified. For some time it has been argued that 
the use of computer-assisted assessment can engage students by providing feedback 
instantly and is therefore closely connected with activity. Often students repeat the 
assessment without the fear of publicly revealing any inadequacies in their knowl¬ 
edge to peers and tutors. Providing students with access to such tests are increas¬ 
ingly popular through easy-to-use online assessment tools and within virtual 
learning environments. Peer-assessment and peer-feedback activities are increasingly 
making use of technology. It is claimed that by providing feedback on their peers’ 
work (often anonymously), students are better equipped to engage with published 
assessment criteria and often make greater improvements on their own work as a 
result of giving feedback than from the feedback they receive themselves from their 
peers. 
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As a result of conducting this review of the literature about the technological 
interventions that might encourage students to engage with their feedback, we rea¬ 
lise that day-to-day effective assessment and feedback practice is rarely reported in 
the literature. It may be that practitioners do not see the value in what they do due 
to its proximity to their practice, and we feel that there is a need for day-to-day 
practice to be reported and celebrated. Often examples of good practice can be seen 
at conferences and small-scale internal events. Where it is published, there tends to 
be a focus on the problematic nature of assessment and feedback, rather than report¬ 
ing on effective strategies as well as where the use of technology can support the 
production, delivery and student engagement with feedback. 


Note 

1. Please note, as a clarifying point this paper will use the term ‘module’ to mean an individ¬ 
ual unit or course of study; and where quoted from third-party sources, the term ‘marks’ 
means ‘grades’. 
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